Kinship Verification from Videos using Spatio-Temporal Texture Features and Deep Learning

Automatic kinship verification using facial images is a relatively new and challenging research problem in computer vision. It consists in automatically predicting whether two persons have a biological kin relation by examining their facial attributes. While most of the existing works extract shallow handcrafted features from still face images, we approach this problem from spatio-temporal point of view and explore the use of both shallow texture features and deep features for characterizing faces. Promising results, especially those of deep features, are obtained on the benchmark UvA-NEMO Smile database. Our extensive experiments also show the superiority of using videos over still images, hence pointing out the important role of facial dynamics in kinship verification. Furthermore, the fusion of the two types of features (i.e. shallow spatio-temporal texture features and deep features) shows significant performance improvements compared to state-of-the-art methods.Comment: 7 page

Contribution à l'analyse de visages à partir d'images RVB et de cartes de profondeur

Automatic human face analysis refers to the processing of facial images by machines in order to infer useful information, such as identity, gender, ethnicity, mood, etc. Face analysis has many interesting applications in security, human computer interaction, social media analysis, etc. Therefore, though face analysis is a well-established computer vision problem, it is still an active research topic attracting considerable attention from researchers. The research community mainly aims to develop more robust systems with the ability to fulfill the requirements of current applications.This thesis contributes to a number of face analysis tasks: face verification and identification, gender recognition, ethnicity recognition and kinship verification. Faces from three different imaging supports i.e. RGB images, depth maps and videos are used throughout the thesis. We present novel approaches and in-depth studies for solving and improving the face analysis problem.First, we tackle face verification problem from RGB images. The local binary patterns based face verification scheme has been revised through proposing novel efficient representations, which cope with the original approach drawbacks while improving the verification performance.Next, the problems of identity, gender and ethnicity recognition are investigated from both RGB and depth images. The aim is to assess the usefulness low-quality depth images, acquired with Microsoft Kinect low-cost sensor, in coping with facial analysis tasks. The performance of RGB images and depth maps are compared to show the ability of the latter ones to deal with sever environment illumination circumstances.Furthermore, the thesis contributes to the problem of kinship verification from videos, where the family relationship between two persons is checked by comparing their facial attributes. The dynamics of faces are efficiently coded by the means of spatio-temporal descriptors and deep features. The value of using videos in kinship problem is shown by comparing their performance against that of still images.Throughout the thesis, various benchmark databases are used and extensive experiments are carried out to validate our proposed approaches and developed methods.Besides, the results of the proposed approaches are compared against the state of the art, highlighting our contributions and showing improvements. Future directions for the presented contributions are outlined at end of the thesis.L'analyse automatique du visage se réfère au traitement des images faciales par les machines afin d'inférer des informations utiles, telles que l'identité, le sexe, l'ethnicité, l'humeur, etc. L'analyse du visage a de nombreuses applications intéressantes en sécurité, interaction homme-machine, analyse des médias sociaux, etc. Par conséquent, bien que l'analyse du visage soit un problème de vision en informatique bien établi, il s'agit toujours d'un sujet de recherche actif qui attire l'attention considérable des chercheurs. La communauté des sceintifiques vise principalement à développer des systèmes plus robustes avec la capacité de répondre aux exigences des applications actuelles.Cette thèse contribue à un certain nombre de tâches d'analyse faciale comme : la vérification et l'identification du visage, la reconnaissance du genre, la reconnaissance ethnique et la vérification de la parenté. Des visages à partir de trois supports d'imagerie différents, i.e. des images RVB, des cartes de profondeur et des vidéos sont utilisés tout au long de la thèse. Nous présentons de nouvelles approches et des études approfondies pour résoudre efficacement le problème de l'analyse du visage.Nous abordons en premier le problème de vérification de visage à partir d'images RVB. Le schéma de vérification du visage basé sur les modèles binaires locaux a été révisé en proposant de nouvelles représentations efficaces qui permettent de faire face aux inconvénients initiaux de l'approche tout en améliorant les performances de vérification.Ensuite, les problèmes d'identité, de sexe et d'origine ethnique sont étudiés à la fois à partir d'images RVB et de cartes de profondeur. L'objectif est d'évaluer l'utilité des images de profondeur de faible qualité, acquises avec le capteur Microsoft Kinect à faible coût, pour faire face aux tâches d'analyse faciale. Les performances des images RVB et des cartes de profondeur sont comparées pour montrer la capacité de ces dernières à faire face à des situations compliqués d'éclairement de l'environnement.En outre, la thèse contribue au problème de la vérification de parenté à partir de vidéos, où la relation de famille entre deux personnes est vérifiée en comparant leurs attributs faciaux. La dynamique des faces est efficacement codée à l'aide de descripteurs spatio-temporels et de traits profonds. L'utilisation de vidéos dans le problème de parenté est privilégiée après une comparaison des performances avec l'utilisation d'images fixes.Tout au long de la thèse, diverses bases de données de référence sont utilisées et des expériences approfondies sont effectuées pour valider les approches proposées et les méthodes développées.Par ailleurs, les résultats des approches proposées sont comparés aux méthodes de référence dans le domaine, en mettant en évidence nos contributions et en montrant des améliorations. Des orientations futures sont présentées à la fin de la thèse

Efficient Tensor-Based 2D+3D Face Verification.

International audienc

3D Face Verification across pose based on Euler rotation and tensors.

International audienc

Kinship verification using color features and extreme learning machine

Abstract Kinship verification from faces is a challenging task that is attracting an increasing attention in the recent years. The proposed methods so far are not robust enough to predict the kin between persons via facial appearance only. The initial studies using deep convolutional neural networks (CNN) have not shown their full potential as well, mainly due to limited training data. To mitigate this problem, we propose a new approach to kinship verification based on color features and extreme learning machines (ELM). While ELM aims to deal with small size training sets, color features are proven to provide significant enhancement over gray-scale counterparts. We evaluate our proposed method on three benchmark and publicly available kinship databases, namely KinFaceW-I, KinFaceW-II and TSKinFace. The obtained results compares favorably against some state-of-the-art methods including those based on deep learning

Efficient Tensor-Based 2D+3D Face Verification.

International audienc

A Tensor Approach for Activity Recognition and Fall Detection Using Wearable Inertial Sensors

International audienc

Kinship verification from face images in discriminative subspaces of color components

International audienceAutomatic facial kinship verification is a challenging topic in computer vision due to its complexity and its important role in many applications such as finding missing children and forensics. This paper presents a Facial Kinship Verification (FKV) approach based on an automatic and more efficient two-step learning into color/texture information. Most of the proposed methods in automatic kinship verification from face images consider the luminance information only (i.e. gray-scale) and exclude the chrominance information (i.e. color) that can be helpful, as an additional cue, for predicting relationships. We explore the joint use of color-texture information from the chrominance and the luminance channels by extracting complementary low-level features from different color spaces. More specifically, the features are extracted from each color channel of the face image and fused to achieve better discrimination. We investigate different descriptors on the existing face kinship databases, illustrating the usefulness of color information, compared with the gray-scale counterparts, in seven various color spaces. Especially, we generate from each color space three subspaces projection matrices and then score fusion methodology to fuse three distances belonging to each test pair face images. Experiments on three benchmark databases, namely the Cornell KinFace, the KinFaceW (I & II) and the TSKinFace database, show superior results compared to the state of the art